
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 1 3- 1 450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/664,682 



09/19/2000 



Suguru Mitsui 



450100-02722 



8097 



20999 7590 03/14/2005 

FROMMER LAWRENCE & HAUG 
745 FIFTH AVENUE- 10THFL. 
NEW YORK, NY 10151 



EXAMINER 



HAN, QI 



ART UNIT 



PAPER NUMBER 



2654 



DATE MAILED: 03/14/2005 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summarv 


Application No. 

09/664,682 


Applicant(s) 

MITSUI, SUGURU 


Examiner 

Qi Han 


Art Unit 

2654 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)13 Responsive to communication(s) filed on 27 September 2004 . 
2a)M This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1,2,5,6,9,10,13 and 14 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1.2.5.6.9.10.13 and 14 is/are rejected. 

7) ! ! C!aim(s) is/are objected to, 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

Response to Amendments 

2. This communication is responsive to the applicants amendment dated 09/27/2004. 
Applicant amended claims 1, 5, 9 and 13, cancelled claims 3-4, 7-8, 11-12 and 15. 

Response to Arguments 

3. Applicant's arguments with respect to independent claim 1 (also applied to claims 5. 9 
and 13) have been considered but are moot in view of the new ground(s) of rejection, since the 
amended claims 1, 5, 9 and 13 introduce new issue(s). 

In response to applicant's arguments regarding rejection claims 2, 6, 10 and 14, that 
challenge the examiner's official notice (amendment: page 7, paragraph 4 to page 8, paragraph 
3), the examiner provides two references for supporting the official notice, as evidenced by 
Burges et al. (5, 647,027) who discloses using "logical operation" on pixels (element) for 
combining images (column 7, lines 45-51), and by Madisetti et al. (The Digital Signal Processing 
Handbook, 1998, ISBN: 0-8493-8572-5) who teaches binary (logical) operations on a 
pixel(element)-by-pixel basis for combining images (pages 51-35 and 51-36). Therefore, it 
would have been obvious to one of ordinary skill in the art at time the invention was made to 
modify White in view of Ohsawa and Gibson by specifically providing logical operation(s), such 
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as OR and AND, on a picture element for composing pictures, as taught by Burges et al. or 
Madisetti et al., for the purpose of offering more data manipulating feature(s) for a visual 
display. 



Claim Rejections - 35 USC § 112 
The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

4. Claims 1, 5, 9 and 13 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Regarding claims 1, 5, 9 and 13, the limitation "a composing method of said frequency 
analyzed picture and said selected picture on the basis of . . renders the claim indefinite 
because it is unclear what is exact meaning of the limitation. As best understand in view of the 
application, the limitation will be interpreted as "a method of composing said frequency 
analyzed picture and said selected picture ..." hereinafter. 

Claim Rejections - 35 USC § 103 

5. Claims 1, 5, 9 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
White (US 5,734,794) in view of Ohsawa (US 4,973,897) and Gibson (US 6,490,359 Bl). 
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As per claims 1 and 5, White discloses a computer based system utilizing cues extracted 
from audio speech to select from among a database of stored image cells to produce synthesized 
animated characters (abstract), comprising: 

"sound signal obtaining means for obtaining a sound signal", (Fig. 2, 'microphone', c A/D 
converter'); 

"sound signal analyzing means for frequency- analyzing said obtained sound signal", (Fig. 
2 and column 6, lines 5-21, 'audio tools', 'extraction tools 240'; column 8, lines 24-55, 'the 
speech (sound signal) is analyzed with respect to frequency and amplitude'); 

"storing means for storing a plurality of pictures", (Fig. 4 and column 7, lines 10-16, 
'frame memory 216 which is the initial image storage location', 'a cell (image) database 212') 
(for claim 5); 

"selected picture switching means that switches and selects one among a plurality of 
registered pictures as a selected picture" (Fig. 2 and column 6, lines 5-21, 'image tools 220', 'cell 
selector 250' used to receive phoneme and emotion information from extraction tools 240, and 
£ to select appropriate image cell (herein corresponding to picture) from the image tools memory 
220'); and 

"composed picture display means that displays a composed picture produced by 
composing" the selected pictures (column 5, lines 32-34, 'the animated faces of the virtual actors 
can be composited (composed) onto the faces of the body actors'). 

Even though White suggests that the speech is frequency-analyzed by using spectral 
components, including frequency and amplitude (column 8, lines 24-55), White fails to disclose 
using the analyzed result to produce a corresponding picture as the clamed "producing a 
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specified frequency-analyzed picture on the basis of a result of said frequency-analysis". 
However, this feature is well known in the art as evidenced by Ohsawa who discloses system 
analyzer (title), comprising 'dividing an audio or other input signal into a plurality of frequency 
bands', 'displaying (producing) the detected signal levels of the divided frequency bands' 
(column 3, lines 5-16), and graphic presentation (interpreted as frequency analyzed picture) of 
the signal with the frequency bands on an LCD display device (Figs, 1-2 and 5). Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
modify White by specifically providing a mechanism to produce graphic presentation for the 
divided frequency bands, as taught by Ohsawa, for the purpose of observing frequency 
characteristics of an audio signal (Ohsawa: abstract) 

Further, even though White discloses capability of composing selected pictures (column 
5, lines 32-34), as stated above, White in view of Ohsawa does not expressly discloses the 
selected picture "on the basis of a level of a first frequency band of the analyzed sound signal 
from said sound signal analyzing means", the composed picture produced by composing "said 
frequency-analyzed picture and said selected picture", and a "means that changes a method of 
composing said frequency-analyzed picture and said selected picture on the basis of a level of a 
second frequency band of the analyzed sound signal from said signal analyzing means". 
However, these features are well known in the art as evidenced by Gibson who discloses a 
method and apparatus for using visual images to mix sound (title), comprising £ the audio signal 
visual image is segmented into portions that correspond to pre-selected frequency ranges 
(bands)' and 'the frequency components (bands) of the selected audio signal are dynamically 
correlated with . . . corresponding segmented portions of the audio signal visual image' (column 3, 
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lines26-30); 'the other visual characteristics of sphere ...are made interdependent with selected 
audio characteristic of source signal: ...correlated to frequency;... correlated to volume or 
amplitude. . . ' and 'different colors may be assigned to . . . different frequency ranges (bands)' 
(column 6, line 60 to column 7, line 51), which suggests the system can select a picture (a picture 
portion) based on the interdependent audio characteristics, such as frequency ranges and 
amplitude; 'transforming each audio signal into an audio signal visual image (correspond to a 
selected picture)', 'generates audio effect images (herein may include a frequency-analyzed 
picture)', 'adjusting the displayed audio effect images (corresponding to a composed picture), 
'changes audio effects added to the audio signal in response to corresponding user adjustments to 
the displayed audio effect image (changes a method of the composing) (Figs. 6-16 and column 2, 
line 31-40); 'visual mixing activities' by using 'mix window', 'effect window', 'EQ widow' in 
variety of combining (composing) methods (column 5, line 60 to column 10, line 67); which 
suggest the combined system has capability of implementing the functionality as claimed. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify White in view of Ohsawa by specifically providing a selected picture based 
on an amplitude of a frequency range, and a mixing (composing) mechanism for composing and 
changing a mixed picture from different pictures or picture portions, as taught by Gibson, for the 
purpose of presenting the intuitive special characteristics of sound with visuals (Gibson: column 
2, lines 14-16). 

As per claim 9, it recites a picture producing method. The rejection is based on the same 
reason described for claim 1, because claim 9 recites same or similar limitation(s) as claim 1. 
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As per claim 13, it recites a program-storing medium. The rejection is based on the same 
reason described for claim 1, because claim 13 recites same or similar limitation(s) as claim 1. 



6. Claims 2, 6, 10 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
White in view of Ohsawa in view of Gibson, and further in view of well known prior art (MPEP 
2144.03). 

As per claim 2 (depending on claim 1), White in view of Ohsawa and Gibson fails to 
specifically disclose that "said composed picture display means produces said composed picture 
by logically operating a picture element of said frequency-analyzed picture and a corresponding 
picture element of said selected picture". However, an official notice is taken that it is well 
known in the ait to provide logical operation(s) on a picture (or image) element for composing 
pictures. Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify White in view of Ohsawa in view of Gibson by specifically 
providing logical operation(s), such as OR and AND, on a picture element for composing 
pictures, for the purpose of offering more data manipulating feature(s) for a visual display. 

As per claim 6 (depending on claim 5), the rejection is based on the same reason 
described for claim 2, because claim 6 recites same or similar limitation(s) as claim 2. 

As per claim 10 (depending on claim 9), the rejection is based on the same reason 
described for claim 2, because claim 10 recites same or similar limitation(s) as claim 2. 

As per claim 14 (depending on claim 13), the rejection is based on the same reason 
described for claim 2, because claim 14 recites same or similar limitation(s) as claim 2. 
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Conclusion 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 . 136(a). 

8. A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 

CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 



9. Please address mail to be delivered by the United States Postal Service (USPS) as 
follows: 

Mail Stop 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 
or faxed to: 

(703) 872-9306, (for formal communications intended for entry) 

Or: 

(703) 872-9306, (for informal or draft communications, and please label 
"PROPOSED" or "DRAFT") 
If no Mail Stop is indicated below, the line beginning Mail Stop should be omitted from 
the address. 

Effective January 14, 2005, except correspondence for Maintenance Fee payments, 
Deposit Account Replenishments (see 1.25(c)(4)), and Licensing and Review (see 37 CFR 5. 1(c) 
and 5.2(c)), please address correspondence to be delivered by other delivery services (Federal 
Express (Fed Ex), UPS, DHL, Laser, Action, Purolater, etc.) as follows: 
U.S. Patent and Trademark Office 
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Customer Window, Mail Stop 



Randolph Building 
Alexandria, VA 223 14 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Qi Han whose telephone numbers is (703) 305-5631. The 
examiner can normally be reached on Monday through Thursday from 9:00 am to 7:00 p.m. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Richemond Dorvil, can be reached on (703) 305-9645. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Inquiries regarding the status of submissions 
relating to an application or questions on the Private PAIR system should be directed to the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free) or 703-305-3028 between the 
hours of 6 a.m and midnight Monday through Friday EST, or by e-mail at: ebc@uspto.gov. For 
general information about the PAIR system, see http://pair-direct.uspto.gov. 



QH/qh 

March 10, 2005 




DAVID D.KNEPPER 
PR/MARY EXAMINER 



